Traditionally, electronic data is stored locally on a user's computer system by means of a data storage resource such as a hard disk drive (HDD) or other storage media. However, the increasing prevalence of data-heavy resources (for example, real-time high definition video) has led to an increased demand for storage capacity.
An increasingly popular area is what is known as “cloud computing”. Cloud computing provides a set of scalable and often virtual resources over a network such as an Ethernet or the Internet. A “cloud” comprises a consolidated storage system having large storage capacity (typically at the multi-petabyte level) which may serve independent customers (e.g. the cloud acts a storage service provider) or business units within an organisation (e.g. the cloud acts as a common corporate data store). In essence, cloud architecture means that the users generally do not own the physical computing resources they use and, instead, purchase usage from a third-party provider in a service-orientated architecture, or access a common corporate data store.
Cloud-type storage service providers are attractive to small to medium sized enterprises which do not typically have the resources to invest in over-provisioned storage infrastructures which will never be used efficiently. In addition, cloud-type services enable a user having multiple devices (e.g. smartphones, tablets, laptops and workstations) to access common stored data without the need to synchronise the data between the individual devices.
Storage service providers offer such users access to the storage services that they require without the need for capital expenditure on hardware and software solutions. In addition, the cost of hardware is becoming increasingly small in comparison to the cost of maintaining and managing a data storage resource. Therefore, this makes the cloud approach even more attractive to businesses. In many cases, service providers provide services in the manner of a utility service and billed, for example, on the basis of the storage resources (e.g. storage space) consumed by the user or on a periodical billing basis.
It is known for the provision of services by a service provider to be covered by service level agreements (SLAs). An SLA is a negotiated agreement between a service provider (or target) offering a service and a client (or initiator) requiring use of the service. The SLA records a common agreement regarding the quality of service (QoS) to be delivered to the client. For example, in the field of data storage provision, the QoS may relate to a particular level of storage capacity or reliability which can be guaranteed by the service provider.
Increasingly, users of a storage resource may wish to access bandwidth-intensive media such as streaming video data. In this regard, a minimum bandwidth is required to provide smooth playback of the streamed video. If the minimum bandwidth is not met or maintained by the storage resource, then there may be pauses in the video playback whilst the required data is obtained from the storage resource. This leads to an unsatisfactory user experience. As a result, some users of a storage resource may prefer to specify a minimum guaranteed bandwidth in addition to a guaranteed volume of storage space.
However, to date, it has been difficult to guarantee bandwidth in known storage resource arrangements. This is because the performance of a given data storage resource is heavily dependent upon the demands placed upon it. If a number of users are using a large proportion of bandwidth of the data storage resource, then the service provider may not be able to meet the particular bandwidth requirements specified by each user. Given the non-deterministic nature of storage resource access, this means that, currently, it is not possible to provide an assurance of a given bandwidth when the data is accessed.
Typically, the only way to circumvent this problem is to heavily over-provision the data storage resource, i.e. to have sufficient spare capacity to ensure that the specified bandwidth requirements are met. However, this approach is wasteful of resources and uneconomical because a significant proportion of the bandwidth available to the data storage resource must be kept free for use during abnormally heavy traffic conditions, and so is rarely used. Consequently, existing service-orientated storage providers can only guard against “worst case” scenarios of abnormally heavy load.
This issue can be mitigated by providing a “throttled” service, A throttled service is one which is bandwidth limited, as opposed to an “unthrottled” service which has no such bandwidth restriction and would, in the absence of competing data transfers, in principle consume all of the bandwidth available. Throttling of user's services may assist in preventing some users from consuming an unfair proportion of the available bandwidth. However, throttling in a general sense merely provides an upper limit on the bandwidth and cannot provide a minimum lower limit which is required in order to guarantee smooth transmission of, for example, video data as discussed above.
Therefore, known storage provision arrangements suffer from a technical problem that bandwidth requirements cannot be efficiently and accurately guaranteed. This means that real-time guarantees on storage resource bandwidth cannot be made without over-provisioning of the storage resource.
“Argon: performance insulation for shared storage servers” Wachs et al., 5th Usenix conference on file and storage technologies (FAST '07) and U.S. Pat. No. 7,917,903 relate to scheduling methods to enable a particular quality of service to be provided. However, these arrangement are unsuitable for parallel distributed file systems in which data is stored over a number of service systems, such as may be found in a cloud-type system.